Visible, NIR, SWIR, and eSWIR urban and rural scene contrast comparison.

The spectral information contained in the reflective imaging bands can be exploited for specific tasks. Whether targeting or mapping, the visible (VIS), near-infrared (NIR), shortwave infrared (SWIR), extended shortwave infrared (eSWIR) bands perform very differently for every application. For any imaging project, high contrast is very important for good imagery. High contrast leads to more recognizable features within a scene and easier identifiable objects. For mapping, good background scene contrast gives prominent features more detail and their locations can be easily identified. For targeting, low background scene contrast reduces clutter, making it easier to detect objects of interest. The VIS, NIR, SWIR, and eSWIR bands are popular reflective bands to design daytime imaging systems for either task. Deciding on which band will have the best contrast for a specific task is one of the first things to study when designing an imaging system. By measuring urban and rural scenes in terms of equivalent reflectivity (ER), a direct comparison of these four bands can show the utility they provide. The systems used to measure scene contrast are designed to have the same spatial resolution and field of view (FOV). With these instantaneous FOV (IFOV) matched systems, the variance and 1D power spectral densities (PSDs) provide a quantitative comparison for the contrast among the four bands. The ER differences and resulting contrast measured among these four bands show that the eSWIR has the highest contrast in both urban and rural scenes.

Comparison of reflective band (Vis, NIR, SWIR, eSWIR) performance in daytime reduced illumination conditions.

Daytime low-light conditions such as overcast, dawn, and dusk pose a challenge for object discrimination in the reflective bands, where the majority of illumination comes from reflected solar light. In reduced-illumination conditions, the sensor signal-to-noise ratio can suffer, inhibiting range performance for detecting, recognizing, and identifying objects of interest. This performance reduction is more apparent in the longer wavelengths where there is less solar light. Range performance models show a strong dependence on cloud type and thickness, as well as time of day across the reflective wavebands. Through an experimental and theoretical analysis of a passive sensitivity- and resolution-matched testbed, we compare Vis (0.4-0.7 µm), NIR (0.7-1 µm), SWIR (1-1.7 µm), and eSWIR (2-2.5 µm) to assess the limiting cases in which reduced illumination inhibits range performance. The time during dawn and dusk is brief yet does show significant range performance reduction for SWIR and eSWIR. Under heavy cloud cover, eSWIR suffers the most at range due to a low signal-to-noise ratio. In cases of severe reduction in illumination, we propose utilizing active illumination or the emissive component of eSWIR to improve the signal-to-noise ratio for various discrimination tasks.

Radiometry and contrast-to-noise ratio for continuous-wave and laser range-gated active imaging systems.

Resolution and sensitivity must be considered in the design of an active imaging system. System sensitivity is characterized by the signal-to-noise or contrast-to-noise ratio and is derived through radiometry. We present a tutorial for the radiometry associated with the contrast-to-noise ratio for active continuous-wave and laser range-gated imaging systems, giving a useful metric for determining reflective-band sensor performance against a target and background. A calculation of the full power and contrast-to-noise ratio terms is shown for an example case, and all relevant radiometric signal terms are covered while describing the assumptions made. Coherent effects on signal-to-noise ratio are excluded from this analysis.

System-level noise performance of coherent imaging systems.

We provide an in-depth analysis of noise considerations in coherent imaging, accounting for speckle and scintillation in addition to "conventional" image noise. Specifically, we formulate closed-form expressions for total effective noise in the presence of speckle only, scintillation only, and speckle combined with scintillation. We find analytically that photon shot noise is uncorrelated with both speckle and weak-to-moderate scintillation, despite their shared dependence on the mean signal. Furthermore, unmitigated speckle and scintillation noise tends to dominate coherent-imaging performance due to a squared mean-signal dependence. Strong coupling occurs between speckle and scintillation when both are present, and we characterize this behavior by fitting a scale factor capable of generating variances in closed form. We verify each of these claims through a series of wave-optics simulations, and we see strong agreement in general between numerical results and theoretical predictions. Our findings allow us to confidently gauge signal-to-noise ratio (SNR) expectations when active illumination produces coherent noise.

Active-tracking scaling laws using the noise-equivalent angle due to speckle.

It is well known to system engineers that speckle imposes a limitation on active-tracking performance, but scaling laws that quantify this limitation do not currently exist in the peer-reviewed literature. Additionally, existing models lack validation through either simulation or experimentation. With these points in mind, this paper formulates closed-form expressions that accurately predict the noise-equivalent angle due to speckle. The analysis separately treats both well-resolved and unresolved cases for circular and square apertures. When compared with the numerical results from wave-optics simulations, the analytical results show excellent agreement to a track-error limitation of (1/3)λ/D, where λ/D is the aperture diffraction angle. As a result, this paper creates validated scaling laws for system engineers that need to account for active-tracking performance.

Mid-wave and long-wave infrared signature model and measurement of power lines against atmospheric path radiance.

The signal to noise ratio and corresponding visibility of power cables as seen by military aircrafts is critical for crew safety. During low altitude operations, rotorcraft systems must be able to navigate these power lines during flight. Many of these military missions are flown at night which means the reflective bands including the visible, near infrared and short-wave infrared do not provide sufficient light. However, the emissive bands of the mid-wave infrared (MWIR) and long-wave infrared (LWIR) can be used to distinguish the location of these wires. LWIR sensors are typically used for pilotage applications. In both the LWIR and MWIR, the signal to noise depends on the wire emissivity and reflectivity as well as the ground and sky background path radiance. The signal to noise ratio is strongly dependent on the elevation of the viewing angle. In this paper, we model the signal to noise ratio as a function of elevation viewing angle using wire reflectivity and emissivity as well as MODTRAN calculations for path radiance. We also take MWIR and LWIR measurements to compare these two bands to the modelling results. We provide a summary of both model and measurements and make conclusions.

Refining Atmosphere Profiles for Aerial Target Detection Models.

Atmospheric path radiance in the infrared is an extremely important quantity in calculating system performance in certain infrared detection systems. For infrared search and track (IRST) system performance calculations, the path radiance competes with the target for precious detector well electrons. In addition, the radiance differential between the target and the path radiance defines the signal level that must be detected. Long-range, high-performance, offensive IRST system design depends on accurate path radiance predictions. In addition, in new applications such as drone detection where a dim unresolved target is embedded into a path radiance background, sensor design and performance are highly dependent on atmospheric path radiance. Being able to predict the performance of these systems under particular weather conditions and locations has long been an important topic. MODTRAN has been a critical tool in the analysis of systems and prediction of electro-optical system performance. The authors have used MODTRAN over many years for an average system performance using the typical "pull-down" conditions in the software. This article considers the level of refinement required for a custom MODTRAN atmosphere profile to satisfactorily model an infrared camera's performance for a specific geographic location, date, and time. The average difference between a measured sky brightness temperature and a MODTRAN predicted value is less than 0.5 °C with sufficient atmosphere profile updates. The agreement between experimental results and MODTRAN predictions indicates the effectiveness of including updated atmospheric composition, radiosonde, and air quality data from readily available Internet sources to generate custom atmosphere profiles.

Wave-optics simulation of dynamic speckle: I. In a pupil plane.

This two-part paper demonstrates the use of wave-optics simulations to model the effects of dynamic speckle. In Part I, we formulate closed-form expressions for the analytical irradiance correlation coefficient, specifically in the pupil plane of an optical system. These expressions are for square, circular, and Gaussian scattering spots and four different modes of extended-object motion, including in-plane and out-of-plane translation and rotation. Using a phase-screen approach, we then simulate the equivalent scattering from an optically rough extended object, where we assume that the surface heights are uniformly distributed and delta correlated from grid point to grid point. For comparison to the analytical irradiance correlation coefficient, we also calculate the numerical irradiance correlation coefficient from the dynamic speckle after propagation from the simulated object plane to the simulated pupil plane. Overall, the analytical and numerical results definitely demonstrate that, relative to theory, the dynamic speckle in the simulated pupil plane is properly correlated from one frame to the next. Such validated wave-optics simulations provide the framework needed to model more sophisticated setups and obtain accurate results for system-level studies.

Wave-optics simulation of dynamic speckle: II. In an image plane.

This two-part paper demonstrates the use of wave-optics simulations to model the effects of dynamic speckle. In Part II, we formulate closed-form expressions for the analytical irradiance correlation coefficient, specifically in the image plane of an optical system. These expressions are for square, circular, and Gaussian limiting apertures and four different modes of extended-object motion, including in-plane and out-of-plane translation and rotation. Using a phase-screen approach, we then simulate the equivalent scattering from an optically rough extended object, where we assume that the surface heights are uniformly distributed and delta correlated from grid point to grid point. For comparison to the analytical irradiance correlation coefficient, we also calculate the numerical irradiance correlation coefficient from the dynamic speckle after propagation from the simulated object plane to the simulated image plane. Overall, the analytical and numerical results definitely demonstrate that, relative to theory, the dynamic speckle in the simulated image plane is properly correlated from one frame to the next. Such validated wave-optics simulations provide the framework needed to model more sophisticated setups and obtain accurate results for system-level studies.

Detection of small targets in the infrared: an infrared search and track tutorial.

Airborne target detection in the infrared has been classically known as infrared search and track or IRST. From a military point of view, it can be described as target detection at long ranges where the target image is subpixel in size. Here, the target is "unresolved." It can also describe the detection of aircraft near the observer using distributed apertures in a spherical detection field. From a commercial point of view, an important application is drone detection near live airport operations. As drones become more common, the dual-use functionality of IRST systems is expanding. Technology improvements for IRST systems include the wide proliferation of infrared staring focal planes. New readout integrated circuits allow for time-delay-integration of large format detectors. Stare-step sensors in the future appear to be as common as gimbal-scanned thermal imagers. Detection probability analysis and IRST sensor design is different than targeting system design. We provide a tutorial here on IRST system calculations as well as discussions on broadband versus spectral calculations and new technology considerations.

Detection of Burmese pythons in the near-infrared versus visible band.

Human task performance studies are commonly used for detecting and identifying potential military threats. In this work, these principles are applied to detection of an environmental threat: the invasive Burmese python. A qualitative detection of Burmese pythons with a visible light camera and an 850 nm near-infrared (NIR) camera was performed in natural Florida backgrounds. The results showed that the difference in reflectivity between the pythons and native foliage was much greater in NIR, effectively circumventing the python's natural camouflage in the visible band. In this work, a comparison of detection performance in the selected near-infrared band versus the visible band was conducted. Images of foliage backgrounds with and without a python were taken in each band in daylight and at night with illumination. Intensities of these images were then calibrated and prepared for a human perception test. Participants were tasked with detecting pythons, and the human perception data was used to compare performance between the bands. The results show that the enhanced contrast in the NIR enabled participants to detect pythons at 20% longer ranges than the use of visible imagery.

Infrared sensor performance with boost and restoration filtering.

The targeting task performance (TTP) model for prediction of target identification range suggests that boost filtering with a well-sampled, low-noise long-wave infrared (LWIR) sensor can substantially increase target ID range (by enhancing contrast at high spatial frequencies). We model a notional high-performance LWIR imaging system with a high F-number, deep electron wells, and a small-pitch focal plane array. System analysis performed with the Night Vision Integrated Performance Model (NVIPM) predicts that a range enhancement upwards of 50% is achievable with Wiener restoration applied to imagery from the modeled sensor. Human perception experiments were performed on simulated target imagery, with range through different boost filters (including a Wiener restoration filter) compared to the no-post-filter case. The TTP model was found to significantly overestimate the performance improvement due to boost and restoration filtering. Alternate predictions based on the Johnson criteria were also performed, and these underestimated the impact of boost. We speculate on reasons for the discrepancy and on promising avenues for future research. Sensor parameters, NVIPM predictions, filter parameters, and experimental data are provided.

Design considerations for advanced MWIR target acquisition systems.

In this paper, mid-wave infrared (MWIR) sensor optimization is provided as a function of the parameter Fλ/d, where F is the f-number, λ is the effective wavelength, and d is the detector pitch. For diffraction limited systems, acquisition range is related to the instantaneous field of view (detector limited operation) when Fλ/d<1, and to the optical properties (optics limited operation) when Fλ/d>2.0. Range performance is a combination of detector and optics resolution limits when Fλ/d is in between. When the system is not strictly diffraction or sampling limited, the optimal Fλ/d depends on other system component characteristics and conditions. Optical system aberrations affect system resolution and decrease range performance. As background shot noise, dark current shot noise, and read noise increase, range decreases. In the infrared spectral region, atmospheric absorption leads to reemission of thermal energy. The detected reemission creates additional shot noise. Atmospheric attenuation greatly affects MWIR sensor range performance. Next-generation MWIR sensors will have smaller detectors, larger arrays, and better sensitivity to enable Fλ/d-based optimization. Previous studies (ΔT=4K for tracked vehicles) suggest that an initial design point is Fλ/d≈2.0. When detecting low contrast targets (ΔT∼0.1K), sensor gain is used to increase the signal for a desired displayed contrast. This gain increases displayed noise and reduces acquisition range. This is typically not an issue for long-wave infrared sensors due to the excess number of photons in the 8-12 µm band but poses a problem for MWIR sensors, which are photon starved. Under such scenarios, the optimum Fλ/d appears to be about 1.5 for MWIR sensors. The results here provide reasonable strategies for MWIR system optimization and a direction associated with future MWIR focal plane development.

Performance comparison of sparse array millimeter wave imager configurations.

Millimeter wave imaging systems are a promising candidate for several applications such as indoor security, industrial non-destructive evaluation, and automotive radar systems. In this paper, we compare the performance of various array configurations that can be enabled by recent automotive radar chips, for imaging applications. High resolution real-time imaging requires an extensive number of measurements which demands a large number of emitters and receivers. Hence, cost and size become major considerations in the design process. In an attempt to reduce the number of emitter and receiver elements, we compare various antenna array architectures to optimize the hardware implementation for high resolution imaging. We consider mono-static single-input-single-output (SISO), multi-static multiple-input-multiple-output (Full-MIMO), and hybrid localized MIMO-SISO (Local-MIMO) architectures. The computationally reconstructed image quality and point spread function from each architecture are compared and traded for the system engineering complexity and cost. We present measurement results from a Synthetic Aperture Radar (SAR) system based on an automotive radar sensor to ensure it is representative of the system's physics. The comparative results of the SISO, Full-MIMO and Local-MIMO simulations provide for affordable alternatives to the high cost SISO approach.

Performance benefits of small-pitch, high-dynamic-range, digitally restored infrared imaging systems using triangle orientation discrimination and NVIPM.

For the past year, the authors have been studying a long-wave infrared (LWIR) sensor design concept that combines high detector well capacity, small-pitch detectors, and digital image processing to optimize target acquisition. Theoretical performance modeling [via the Night Vision Integrated Performance Model (NVIPM)] suggests that our approach offers a large increase in target identification range, but multiple field trials using triangle orientation discrimination (TOD) have yielded results that are inconsistent with the model's predictions. For this reason, we have performed human perception experiments on simulated TOD targets, with and without image processing, to assess the utility of our approach and the value of TOD as an evaluation for digital image enhancement. The results do not agree with the NVIPM modeling of range improvement. We present our results, along with our modeling and experiment methodologies, to guide future investigations into the effects of boost or restoration filtering on target identification performance.

Burmese python target reflectivity compared to natural Florida foliage background reflectivity.

The Florida Everglades is infested with Burmese pythons caused by the release of exotic pets in the 1980s. The current estimates are between 30,000 and 300,000 pythons, where the result is a severe decline in Everglade mammals: 90% reductions in raccoon, opossum, bobcats, and foxes. The marsh rabbits are completely gone. The population of the pythons is rapidly increasing exponentially with 20-50 eggs per snake with a life span of up to 20 years. Pythons have been captured in the Everglades with lengths of nearly 6 m. Researchers in the state of Florida are concerned that these pythons are (1) permanently damaging the Everglades, (2) migrating further north into populated areas of Florida, and (3) endangering wildlife, pets, and eventually, people. There have been a number of sensing efforts attempted in the large-area detection of pythons, where limited success has been achieved. For example, infrared sensors have been applied to the problem, but the pythons are cold-blooded, so the infrared bands do not work well. Imec has leveraged its expertise and infrastructure in semiconductor processing to produce highly compact, higher performance, and relatively cheaper hyperspectral image sensors and camera systems. In this work, Imec teamed with the University of Florida and Extended Reality Systems to obtain hyperspectral reflectivity measurements of Burmese pythons along with natural Florida background foliage to determine bands or band combinations that may be exploited in the large-area detection of pythons. The bands investigated are the visible-near infrared (or VisNIR) and the shortwave infrared (SWIR) bands. The results show that there are enough differences in the data collection such that a single band, inexpensive VisNIR band camera may provide reasonable results and a two-band, VisNIR/SWIR combination may provide higher performance results. In this paper, we provide the VisNIR results.

Predicting range performance of sampled imagers by treating aliased signal as target-dependent noise.

This paper presents a new theory to predict the impact of sampling on target acquisition. The aliased signal that results from sampling is treated as noise. The aliased signal is different from detector noise in two ways. First, aliasing disappears as the target contrast decreases. Second, the image corruption due to aliasing gets worse with increased range. This is because sampling is constant in angle space, and targets become poorly sampled as range increases. The theory is presented, along with the results of three experiments. The match between model and experiment is excellent.

Predicting the probability of facial identification using a specific object model.

Many security and surveillance tasks involve either finding an object in a cluttered scene or discriminating between like objects. For example, an observer might look for a person of known height and weight in a crowd, or he might want to positively identify a specific face. The paper "Modeling target acquisition tasks associated with security and surveillance" [Appl. Opt. 46, 4209 (2007)] describes a specific-object model used to predict the probability of accomplishing this type of task. We describe four facial identification experiments and apply the specific-object model to predict the results. Facial identification is accurately predicted by the specific-object model.

Midwave infrared and visible sensor performance modeling: small craft identification discrimination criteria for maritime security.

The new emphasis on Anti-Terrorism and Force Protection (AT/FP), for both shore and sea platform protection, has resulted in a need for infrared imager design and evaluation tools that demonstrate field performance against U.S. Navy AT/FP requirements. In the design of infrared imaging systems for target acquisition, a discrimination criterion is required for successful sensor realization. It characterizes the difficulty of the task being performed by the observer and varies for different target sets. This criterion is used in both assessment of existing infrared sensor and in the design of new conceptual sensors. We collected 12 small craft signatures (military and civilian) in the visible band during the day and the long-wave and midwave infrared spectra in both the day and the night environments. These signatures were processed to determine the targets' characteristic dimension and contrast. They were also processed to band limit the signature's spatial information content (simulating longer range), and a perception experiment was performed to determine the task difficulty (N50 and V50). The results are presented and can be used for Navy and Coast Guard imaging infrared sensor design and evaluation.

Characteristics of infrared imaging systems that benefit from superresolution reconstruction.

There have been numerous applications of superresolution reconstruction algorithms to improve the range performance of infrared imagers. These studies show there can be a dramatic improvement in range performance when superresolution algorithms are applied to undersampled imager outputs. These occur when the imager is moving relative to the target, which creates different spatial samplings of the field of view for each frame. The degree of performance benefit is dependent on the relative sizes of the detector/spacing and the optical blur spot in focal plane space. The minimum blur spot size achievable on the focal plane is dependent on the system F/number. Hence, we provide a range of these sensor characteristics, for which there is a benefit from superresolution reconstruction algorithms. Additionally, we quantify the potential performance improvements associated with these algorithms. We also provide three infrared sensor examples to show the range of improvements associated with provided guidelines.